September 30, 2016 / by Adam Reynolds /

Physics exam questions

We have started to adapt exam questions into dynamic questions on Siyavula Practice and we’re pretty excited about this. The process is an interesting one and we thought we’d share the approach taken by our Education Technology team in producing one of these questions. In this post we’ll take a look at a specific example of a typical exam question that we’ve adapted. We’ll expose how we handle definitions, questions and diagram drawing. We’ll also take a look at some of the thinking that goes into incorporating variability into the questions as well as the philosophy behind our worked solutions.

Let’s get going!

A classic structure for exam questions

As far as possible, we have attempted to retain the same structure and scaffolding that a learner can expect to find in an exam. During the process of adapting a static question to something that is dynamic and variable, we try not to make changes that will fundamentally change the physics skills and content assessed. Working online means working within different constraints and, for certain types of questions, we have had to come up with creative ways to assess learners in meaningful ways that remain true to the learning objectives. Examples of questions that require creative handling are questions that ask learners to state a law, or draw a diagram.

The following is one of the classic structures found in many exam questions:

• Ask the learner to state a definition or a law. A knowledge question like this often acts as a clue for subsequent questions.
• Ask a series of questions of increasing complexity. These questions usually build on each other to provide some form of scaffolding for what could have been a single more complicated question. These questions are usually at the comprehension and application level.
• Ask a short higher-order question that is intended to probe a deeper understanding of the content. This question will often require some level of analysis, evaluation or synthesis.

Question 2 in the Department of Basic Education (DBE) November 2014 Physics Paper is an example of an exam question that follows such a pattern.

We have made this question the basis for one of our variable exercises on Siyavula Practice. Throughout the rest of this blog post we’ll be talking through the process we took to adapt this particular question for our practice platform and we’ll refer back to this question frequently.

Definitions and laws on Siyavula Practice

We all know that definitions and laws are foundational to science: words in science have precise meanings, and as a result mastering terminology is a crucial component skill in mastering science.

On Siyavula Practice we do not ask learners to write out a definition for a particular term or the statement of a law on Siyavula Practice in the same way that an exam question might. The questions on our platform are designed to aid learning and mastering concepts and as such, the questions are more formative rather than summative.

We identify the same key elements that an examiner will be looking for in the statement and then ask learners to construct the definition using drop-down menus.

For example, here is the statement of Newton’s Second Law of motion from the memo for Question 2.1 with the essential components (according to the examiner) underlined.

Below is the equivalent question on Siyavula Practice:

These statement and definition questions are not trivial either. The distractors have been carefully selected to match common errors, yet still remain grammatically correct. Being made to construct the definition from plausible distractors offers a valuable learning opportunity because learners really need to engage with the content to get the question right.

For example, learners often confuse the direction of acceleration with the direction of movement. Similarly, acceleration and velocity (or speed) are often confused with each other. And terms such as “directly proportional” and “inversely proportional” are often misunderstood. The following example shows how this series of common errors may be marked by our software.

When learners try a different version of these terminology questions they may be presented with a different set of distractors or they may be required to supply a different part of the definition or law being tested.

It should be noted that the definitions and laws we have used are compatible with, but not limited by, both the DBE and Independent Examination Board (IEB) Subject Assessment Guidelines (SAG) documents.

Diagram identification on Siyavula Practice

Another key skill in science is graph and diagram drawing. Drawing a free-body or force diagram showing all forces acting on an object is a useful way of summarising the interactions in a system. Doing this (whether specifically asked to or not) is an incredibly helpful way of getting into the problem as well as in choosing appropriate physical laws or principles to apply.

At the moment we are unable to ask learners to actually draw free-body or force diagrams, but we can present them with a carefully crafted set of diagrams from which to choose. Our version of Question 2.2 reads:

“Which one of following diagrams is a correctly labelled free-body diagram indicating all the forces acting on the 5 kg block?”

In order to try and make this type of question as formative as possible, our solutions, unlike those in typical memos for multiple choice questions, do not just give the correct answer. They explain why the distractors are incorrect and also explain why the correct option is correct

Multiple choice questions can be devious and a well-crafted MCQ will present compelling distractors for the unwary learner. The confusion often induced by a typical exam-level MCQ needs to be addressed with immediate feedback so that learners can reinforce the correct concepts.

Variability in the question

When an examiner sets a static exam question they have to make certain choices. In the exam question that we’ve been looking at, the examiner may have had to make the following choices:

1. Should the block with the greater mass go on the top or the bottom?
2. Should the system be accelerating up or down?
3. Should the diagram show the hanging blocks on the left or the right?

The last option may be classed as purely aesthetic, but sometimes it can be important to have this variety so that learners do not get used to always seeing things from the same perspective. Learners need to get used to working with the unfamiliar and having your inclined plane always slope up from bottom-left to top-right will not necessarily help them achieve this subtle skill.

One of the great things about making variable questions for Siyavula Practice is that we do not have to make any of the above choices! Not being required to discard good options for the question simply based on needing to “make a choice” allows us to incorporate the subtle nuances that help to make each version of the question unique.

Below we show the diagrams of two different versions of this question in which opposite decisions were made for all of the above three choices:

In both of the above versions of the question, learners will be practising the same core concept. However, each scenario results in a different outcome. In one case they’ll be applying Newton’s Second Law to each object in the system and they’ll work out that the diagram is accelerating downwards. In the other case, they’ll still apply Newton’s Second Law to both blocks, but they’ll discover that the blocks are accelerating upwards.

Incorporating variability into our questions gives learners who elect to see another version of the question a more meaningful learning experience. They can feel comfortable that they will practise the same set of skills, but they need to stay on their toes when working their way through the new version of the question.

Our worked solutions

In the same way that a diligent learner will work carefully through the memo when doing traditional “past-paper work”, we expect learners using Siyavula Practice to make thorough use of our worked solutions. Naturally, our solutions track the variability in the questions - a slightly different question will come with an answer that perfectly matches the new version. The solutions are also designed to offer far more than a typical exam memo or solution from a past-paper question.

For example, Question 2.4 in the exam question we’re working through asks which string will break first when the horizontally applied force is replaced with a sharp pull. Here, our worked solution gives a careful explanation that helps learners to understand why the upper string breaks first. Compare this to the memo, which simply states the conclusion: perhaps not the best option for a learner who is struggling to engage with the concepts involved, or who selected the incorrect answer for reasons that they considered to be sensible.

The approaches taken in our worked solutions are in line with best-practice (in both physics and physics education) as well as following the methods prescribed by examining bodies. In the question we’ve been working through, we solve for the tension in the rope connecting the two blocks and the acceleration of the system by applying Newton’s Second Law of motion to each block separately and then making use of simultaneous equations, rather than treating the system as one object. This is the method prescribed by examiners, and with good reason - it represents far more principled physics.

Another example is drawn from Question 2.3. For more computational questions, learners are exposed to all the steps that they need to show in exams to get marks. In addition, these steps are interspersed with explanations written in simple English - our objective is to aid learners’ understanding of the physics concepts, not test their language fluency.

The ability to take carefully constructed exam questions that test students understanding of core concepts, and vary a number of parameters in an automated fashion, means that learners can be exposed to many, varied examples of the same basic question. Coupled with fully worked, step-by-step solutions and explanatory diagrams, this gives them the opportunity to make sure they understand the concepts being tested and ultimately practise for true mastery.

Exams are just around the corner. Bring them on!